Uncovering feature interdependencies in high-noise environments with stepwise lookahead decision forests

Abstract Conventionally, random forests are built from “greedy” decision trees which each consider only one split at a time during their construction. The sub-optimality of greedy implementation has been well-known, yet mainstream adoption more sophisticated tree building algorithms lacking. We examine under what circumstances an less actually yields outperformance. To this end, “stepwise lookahead” variation the forest algorithm is presented for its ability to better uncover binary feature interdependencies. In contrast approach, included in algorithm, simultaneously three nodes tiers depth two. It demonstrated on synthetic data and financial price series that lookahead version significantly outperforms when (a) certain non-linear relationships between feature-pairs present (b) if signal-to-noise ratio particularly low. A long-short trading strategy copper futures then backtested by training both stepwise predict signs daily returns. resulting superior performance least partially explained presence “XOR-like” long-term short-term technical indicators. More generally, across all examined datasets, no such features present, similar. Given enhanced understand feature-interdependencies complex systems, useful extension toolkit scientists, particular machine learning, where conditions typically met.